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DETAILED ACTION 
Continued Examination Under 37 CFR 1. 1 14 

1. A request for continued examination under 37 CFR 1.114, including tine fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
09/14/2010 has been entered. 

Response to Arguments 

2. Applicant's arguments filed 09/14/2010 have been considered but are moot in 
view of the new ground(s) of rejection. 

Ciaim Objections 

3. Claims 24, 25, 32, 33, are objected to because of the following informalities: 
claims 24 and 25 depend on cancelled claim 23 while claims 32 and 33 depend on 
cancelled claim 31 . Through out this Office Action claims 24 and 25 will depend from 
claim 22 while claims 32 and 33 depend from claim 30. Appropriate correction is 
required. 

Claim Rejections - 35 USC §103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Application/Control Number: 10/637,625 
Art Unit: 2423 



Page 3 



5. Claims 1-5, 7-11, 13-17, 22, 24-25, 30, 32-33 are rejected under 35 U.S.C. 

103(a) as being unpatentable over US Patent 5,774,666 to Portuesi (Portuesi), in view 

of US Patent 6,604,242 to Weinstein et al. (Weinstein). 

Regarding claim 1 , Portuesi teaches elements stored in a non-transitory 
computer-readable memory (figure 1, label 6; col . 4, II. 19-21) wherein the 
elements are structured for use by a computer to display (figure 1, label 12; 
because 10 is in a computer see col. 4, II. 34-36) a multimedia presentation 
(video files) and a visual cue (url's) on a display of the computer (figure 1 , label 
12), wherein the elements comprise at least: a multimedia presentation element 
(video); and a visual cue element (uri) which is nested within the multimedia 
presentation element (url's are embedded (nested) in video files see figures 2-3; 
col. 4, II. 21-23); wherein the multimedia presentation element includes access 
information for the multimedia presentation (col. 1, II. 34-40; col. 4, II. 46-54 
(video files include temporal information)); and wherein the visual cue element, 
which is nested within the multimedia presentation element, includes (figures 2-3; 
col. 4, II. 21-23): a visual attribute that defines a visual representation of the 
visual cue (figure 4, 40 (see circular shape of hot spot)); a spatial attribute that 
defines, in spatial relation to a spatial position of the multimedia presentation on 
the display , spatial characteristics of the visual cue (col. 5, II. 20-28, 34-36); and 
a temporal attribute that defines, in temporal relation to a temporal progression of 
the multimedia presentation on the displav (col. 3, II. 29-34; col. 6, II. 43-45), 
temporal characteristics of the visual cue, wherein the computer uses the 
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multimedia presentation element and the nested visual cue element ( figures 2-3; 
col. 4, II. 21-23) to access the multimedia presentation and to synchronously 
display on the display the visual representation of the visual cue on over the 
multimedia presentation (figure 4, label 40; figure 3, labels 31, 32; col. 4, II. 24- 
28), during a period of time which is based on the temporal characteristics of the 
visual cue as defined in the temporal attribute and on the temporal progression of 
the multimedia presentation (col. 4, II. 49-67; col. 5, II. 1-16; col. 3, II. 2-34; col. 6, 
II. 43-45), and at a location which is based on the spatial characteristics of the 
visual cue as defined in the spatial (col. 5, II. 20-28, 34-36) and on the spatial 
position of the multimedia presentation (col. 1, II. 34-40; col. 4, II. 49-53; col. 5, 
20-28). 

But Portuesi is silent on displaying multimedia presentation and a visual 
cue using XML-based elements. 

Weinstein teaches combining television broadcast and personalized 
/interactive information (title) wherein XML can be used to synchronize the links 
or hyperlinks (personalized/interactive information with the television broadcast 
(Abstract; col. 1 , II. 52-64) which meets "displaying multimedia presentation and a 
visual cue using XML-based elements". 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the media player application of 
Portuesi to include displaying multimedia presentation and a visual cue using 
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XML-based elements as taught by Weinstein in order to support multiple data 
formats from mapping readily existing data structures. 

Regarding claim 2, Portuesi in view of Weinstein teaches everything in 
claim 1 , in addition Portuesi teaches wherein the temporal characteristics include 
at least one of begin time, end time, and duration (col. 5, II. 38-40). 

Regarding claim 3, Portuesi in view of Weinstein teaches everything in 
claim 1 , in addition Portuesi teaches wherein the visual representation includes 
color (col. 6, II. 20 (highlighting renders color obvious)). 

Regarding claim 4, Portuesi in view of Weinstein teaches everything in 
claim 1 , in addition Portuesi teaches wherein the visual representation includes 
shape (figure 4, label 40 (circular)). 

Regarding claim 5, Portuesi in view of Weinstein teaches everything in 
claim 1, in addition Portuesi teaches wherein the spatial characteristics include 
position (col. 5, II. 24). 

Regarding claim 7, Portuesi teaches a browser (figure 5, label 76) that 
displays a multimedia presentation and a visual cue on a display of a computer to 
a user (col. 9, II. 29-38), a method for processing elements, wherein the elements 
comprise : a multimedia presentation element (video): and a visual cue element 
(uri) which is nested within the multimedia presentation element (url's are 
embedded (nested) in video files see figures 2-3; col. 4, II. 21-23); wherein the 
multimedia presentation element includes access information for the multimedia 
presentation (col. 1, II. 34-40; col. 4, II. 46-54 (video files include temporal 



Application/Control Number: 10/637,625 Page 6 

Art Unit: 2423 

information)); and wherein the visual cue element, wliicli is nested witliin tlie 
multimedia presentation element, includes (figures 2-3; col. 4, II. 21-23): a visual 
attribute that defines a visual representation of the visual cue (figure 4, 40 (see 
circular shape of hot spot)); a spatial attribute that defines, in spatial relation to a 
spatial position of the multimedia presentation on the display, spatial 
characteristics of the visual cue (col. 5, II. 20-28, 34-36); and a temporal attribute 
that defines, in temporal relation to a temporal progression of the multimedia 
presentation on the display (col. 3, II. 29-34; col. 6, II. 43-45), temporal 
characteristics of the visual cue (col. 5, II. 20-28, 34-36), wherein the method 
comprises: receiving the elements including the multimedia presentation element 
and the nested visual cue element (figures 2-3; col. 4, II. 21-23); and using the 
multimedia presentation element and the nested visual cue element to access 
the multimedia presentation and to synchronously display on the display the 
visual representation of the visual cue superimposed over the multimedia 
presentation (col. 5, II. 12-16; figure 3, labels 31, 32), during a period of time 
which is based on the temporal characteristics of the visual cue as defined in the 
temporal attribute and on the temporal progression of the multimedia 
presentation (col. 1, II. 29-34), and at a location which is based on the spatial 
characteristics of the visual cue as defined in the spatial attribute (col. 5, II. 20-28, 
34-36) and on the spatial position of the multimedia presentation (col. 1, II. 34-40; 
col. 3, II. 29-34; col. 4, II. 49-53; col. 5, 20-28). 
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But Portuesi is silent on an XIVIL-based browser for displaying multimedia 
presentation and a visual cue using XML-based elements. 

Weinstein teaches combining television broadcast and personalized 
/interactive information (title) wherein XML can be used to synchronize the links 
or hyperlinks (personalized/interactive information with the television broadcast 
(Abstract; col. 1 , II. 52-64) which meets "an XML-based browser for displaying 
multimedia presentation and a visual cue using XML-based elements". 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the browser of Portuesi to include an 
XML-based browser for displaying multimedia presentation and a visual cue 
using XML-based elements as taught by Weinstein in order to support multiple 
data formats from mapping readily existing data structures. 

Regarding claim 8, Portuesi in view of Weinstein teaches everything in 
claim 7, in addition Portuesi teaches wherein the temporal characteristics include 
at least one of begin time, end time, and duration (col. 5, II. 38-40). 

Regarding claim 9, Portuesi in view of Weinstein teaches everything in 
claim 7, in addition Portuesi teaches wherein the visual representation includes 
color (col. 6, II. 20 (highlighting renders color obvious)). 

Regarding claim 10, Portuesi in view of Weinstein teaches everything in 
claim 7, in addition Portuesi teaches wherein the visual representation includes 
shape (figure 4, label 40 (circular)). 
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Regarding claim 1 1 , Portuesi in view of Weinstein teaches everything in 
claim 7, in addition Portuesi teaches wherein the spatial characteristics include 
position (col. 5, II. 24). 

Regarding claim 13, Portuesi teaches a non-transitory computer-readable 
storage medlum(flgure 1, label 6; col . 4, II. 19-21) storing computer executable 
process steps to display a multimedia presentation (video) and a visual cue (uri) 
on a display of a computer to a user (figure 1, label 12; because 10 is in a 
computer see col. 4, II. 34-36), and to process elements (video and urI), wherein 
the elements comprise: a multimedia presentation element (video); and a visual 
cue element (urI) which is nested within the multimedia presentation element 
(url's are embedded (nested) in video files see figures 2-3; col. 4, II. 21-23); 
wherein the multimedia presentation element includes access information for the 
multimedia presentation (col. 1, II. 34-40; col. 4, II. 46-54 (video files include 
temporal information)); and wherein the visual cue element (url). which is nested 
within the multimedia presentation element ( col. 4, II. 21-23) . includes : a visual 
attribute that defines a visual representation of the visual cue (figure 4, 40 (see 
circular shape of hot spot)); a spatial attribute that defines, in spatial relation to a 
spatial position of the multimedia presentation on the display, spatial 
characteristics of the visual cue (col. 5, II. 20-28, 34-36); and a temporal attribute 
that defines, in temporal relation to a temporal procression of the multimedia 
presentation on the display (col. 1, II. 29-34), temporal characteristics of the 
visual cue (col. 5, II. 20-28, 34-36), wherein the computer-executable process 
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steps cause the computer to execute process steps comprising: receiving tlie 
elements including the multimedia presentation element (video: figure 5; col. 9, II. 
21-38) and the nested visual cue element (uri embedded in video file-see figure 
2); and using the multimedia presentation element and the nested visual cue 
element to access the multimedia presentation and to synchronously display on 
the display the visual representation of the visual cue superimposed over the 
multimedia presentation (figure 1 , labels 31 ,32), during a period of time which is 
based on the temporal characteristics of the visual cue as defined in the temporal 
attribute (col. 5, II. 20-28, 34-36) and on the temporal progression of the 
multimedia presentation (col. 3, II. 29-34), and at a location which is based on the 
spatial characteristics of the visual cue as defined in the spatial attribute (col. 5, 
II. 24) and on the spatial position of the multimedia presentation (col. 1, II. 34-40; 
col. 4, II. 49-53; col. 5, 20-28). 

But Portuesi is silent on displaying multimedia presentation and a visual 
cue using XML-based elements. 

Weinstein teaches combining television broadcast and personalized 
/interactive information (title) wherein XML can be used to synchronize the links 
or hyperlinks (personalized/interactive information with the television broadcast 
(Abstract; col. 1, II. 52-64) which meets "displaying multimedia presentation and a 
visual cue using XML-based elements". 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the media player application of 
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Portuesi to include displaying multimedia presentation and a visual cue using 
XML-based elements as taught by Weinstein in order to support multiple data 
formats from mapping readily existing data structures. 

Regarding claim 14, Portuesi in view of Weinstein teaches everything in 
claim 13, in addition Portuesi teaches wherein the temporal characteristics 
include at least one of begin time, end time, and duration (col. 5, II. 38-40). 

Regarding claim 15, Portuesi in view of Weinstein teaches everything in 
claim 13, in addition Portuesi teaches wherein the visual representation includes 
color (col. 6, II. 20 (highlighting renders color obvious)). 

Regarding claim 16, Portuesi in view of Weinstein teaches everything in 
claim 13, in addition Portuesi teaches wherein the visual representation includes 
shape (figure 4, label 40 (circular)). 

Regarding claim 17, Portuesi in view of Weinstein teaches everything in 
claim 13, in addition Portuesi teaches wherein the spatial characteristics include 
position (col. 5, II. 24). 

Regarding claim 22, Portuesi teaches a method for displaying a 
multimedia presentation (figure 1, label 31) and a visual cue (figure 1, label 32) 
on a display of a computer (col. 9, II. 29-38) executing a browser (figure 5, label 
76) which processes elements (video and uri), wherein the elements comprise: a 
multimedia presentation element (video); and a visual cue element (urI) which is 
nested within the multimedia presentation element (url's are embedded (nested) 
in video files see figures 2-3; col. 4, II. 21-23); wherein the multimedia 
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presentation element includes access information for the multimedia presentation 
(col. 1, II. 34-40; col. 4, II. 46-54 (video files include temporal information)); and 
wherein the visual cue element (uri), which is nested within the multimedia 
presentation element (figures 2-3; col. 4, II. 21-23), includes: a visual attribute 
that defines a visual representation of the visual cue (figure 4, 40 (see circular 
shape of hot spot)); a spatial attribute that defines, in spatial relation to a spatial 
position of the multimedia presentation on the display, spatial characteristics of 
the visual cue (col. 1, II. 34-40; col. 5, II. 20-28, 34-36); and a temporal attribute 
that defines, in temporal relation to a temporal progression of the multimedia 
presentation on the display (col. 5, II. 20-28, 34-36; col. 1, II. 34-40; col. 4, II. 46- 
54) . temporal characteristics of the visual cue (col. 5, II. 20-28, 34-36), wherein 
the method comprises: receiving the elements including the multimedia 
presentation element and the nested visual cue element (figures 2-3; col. 4, II. 
21-23); and using the multimedia presentation element and the nested visual cue 
element (figures 2-3; col. 4, II. 21-23) to access the multimedia presentation and 
to synchronously display on the display the visual representation of the visual 
cue superimposed over the multimedia presentation (col. 5, II. 12-16; figure 3, 
labels 31, 32), during a period of time which is based on the temporal 
characteristics of the visual cue as defined in the temporal attribute and on the 
temporal progression of the multimedia presentation (col. 3, II. 29-34), and at a 
location (col. 5, II. 20-28, 34-36) which is based on the spatial characteristics of 
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the visual cue as defined in the spatial attribute and on the spatial position of the 
multimedia presentation (col. 1, II. 34-40; col. 4, II. 49-53; col. 5, 20-28). 

But Portuesi is silent on an XML-based browser for displaying multimedia 
presentation and a visual cue using XML-based elements. 

Weinstein teaches combining television broadcast and personalized 
/interactive information (title) wherein XML can be used to synchronize the links 
or hyperlinks (personalized/interactive information with the television broadcast 
(Abstract; col. 1 , II. 52-64) which meets "an XML-based browser for displaying 
multimedia presentation and a visual cue using XML-based elements". 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the method of Portuesi to include an 
XML-based browser for displaying multimedia presentation and a visual cue 
using XML-based elements as taught by Weinstein in order to support multiple 
data formats from mapping readily existing data structures. 

Regarding claim 24, Portuesi in view of Weinstein teaches everything in 
claim 22, in addition Portuesi teaches a method wherein the temporal 
characteristics include at least one of begin time, end time, and duration (col. 5, 
II. 38-40). 

Regarding claim 25, Portuesi in view of Weinstein teaches everything in 
claim 22, in addition Portuesi teaches a method wherein the visual representation 
includes at least one of a shape (figure 4, label 40 (circular)) and a color (col. 6, 
II. 20 (highlighting renders color obvious)). 
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Regarding claim 30, Portuesi teaches an apparatus comprising: a display 
(12); a computer-readable storage medium (col. 4, II. 19-21) for storing computer- 
executable process steps that cause a multimedia presentation and a visual cue 
to be displayed on the display (col. 9, II. 29-38; figure 3, labels 31, 32), and for 
storing data elements (col. 4, II. 19-21 (storing video files including embedded 
uri's); and a processor (figure 5, label 72) to execute the process steps stored in 
the storage medium (figure 1, label 12); wherein the elements comprise: a 
multimedia presentation element (video); and a visual cue element (uri) which is 
nested within the multimedia presentation element (figures 2-3; col. 4, II. 21-23); 
wherein the multimedia presentation element includes access information for the 
multimedia presentation (col. 1, II. 34-40; col. 4, II. 46-54 (video files include 
temporal information)); and wherein the visual cue element (urI), which is nested 
within the multimedia presentation element (figures 2-3; col. 4, II. 21-23), 
includes: a visual attribute that defines a visual representation of the visual cue 
(figure 4, 40 (see circular shape of hot spot)); a spatial attribute that defines, in 
spatial relation to a spatial position of the multimedia presentation on the displav. 
spatial characteristics of the visual cue (col. 1, II. 31-40; col. 5, II. 20-28, 34-36); 
and a temporal attribute that defines, in temporal relation to a temporal 
progression of the multimedia presentation on the display, temporal 
characteristics of the visual cue (col. 5, II. 20-28, 34-36), and w herein the process 
steps comprise: receivino the elements (figure 5, label 66) including the 
multimedia presentation (video) element and the nested visual cue element 
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(figures 2-3; col. 4, II. 21-23); and using the multimedia presentation element and 
the nested visual cue element to access the multimedia presentation and to 
synchronously display on the display the visual representation of the visual cue 
superimposed over the multimedia presentation (figure 3, labels 31 , 32), durinc a 
period of time which is based on the temporal characteristics of the visual cue as 
defined in the temporal attribute and on the temporal progression of the 
multimedia presentation (col. 3, II. 29-34), and at a location (col. 5, II. 20-28, 34- 
36) which is based on the spatial characteristics of the visual cue as defined in 
the spatial attribute and on the spatial position of the multimedia presentation 
(col. 1, II. 31-40; col. 4, II. 49-53; col. 5, 20-28). 

But Portuesi is silent on an XML-based browser for displaying multimedia 
presentation and a visual cue using XML-based elements. 

Weinstein teaches combining television broadcast and personalized 
/interactive information (title) wherein XML can be used to synchronize the links 
or hyperlinks (personalized/interactive information with the television broadcast 
(Abstract; col. 1 , II. 52-64) which meets "an XML-based browser for displaying 
multimedia presentation and a visual cue using XML-based elements". 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the apparatus of Portuesi to include 
an XML-based browser for displaying multimedia presentation and a visual cue 
using XML-based elements as taught by Weinstein in order to support multiple 
data formats from mapping readily existing data structures. 
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Regarding claim 32, Portuesi in view of Weinstein teaches everything in 
claim 30, in addition Portuesi teaches a method wherein the temporal 
characteristics include at least one of begin time, end time, and duration (col. 5, 
II. 38-40). 

Regarding claim 33, Portuesi in view of Weinstein teaches everything In 
claim 30, In addition Portuesi teaches a method wherein the visual representation 
includes at least one of a shape (figure 4, label 40 (circular)) and a color (col. 6, 
II. 20 (highlighting renders color obvious)). 

Conclusion 

Any Inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANTHONY BANTAMOl whose telephone number is 
(571)270-3581 . The examiner can normally be reached on 8am-5pm Monday-Friday 
(Off every other Friday). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Koenig can be reached on (571)2727296. The fax phone number 
for the organization where this application or proceeding is assigned is 571 -273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/ANTHONY BANTAMOl/ 
Examiner, Art Unit 2423 

/Andrew Y Koenig/ 

Supervisory Patent Examiner, Art Unit 2423 



